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ABSTRACT 



This final report describes results of a planning grant for 
the "Flashlight” project, an effort to develop, test, and disseminate 
evaluation procedures that a postsecondary institution or department could 
use to periodically assess its evolving educational strategies, in particular 
strategies supported by use of computing, video, and telecommunications. This 
planning grant supported the project's first phase: discussion among five 
potential "testbed" institutions, the potential contractor, and the 
Annenberg/Corporation for Public Broadcasting (CPB) Project . Discussion was 
initiated via e-mail and a Delphi survey, and concluded with a two-day 
meeting. The group concluded that Flashlight should develop five families of 
procedures. The first three would help institutions monitor their ability to 
meet the following challenges: (1) improving learning outcomes for people 

completing courses of study; (2) extending enrollment and retention; and (3) 
controlling the spiraling costs of education. The group also proposed that 
Flashlight develop evaluative procedures to monitor changes in 
teaching- learning strategies that are especially well supported by uses of 
computing, video, and telecommunications, and to monitor changes in faculty 
roles and attitudes. The project was ultimately fully approved by the 
Annenberg/CPB Project. Information is provided on the project's purpose, 
background, description, and conclusions. Appendices include the Delphi 
instrument used. (Contains 26 ref erences . ) (DB) 
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Summary 

The Flashlight Project is developing, testing and disseminating evaluation 
procedures that a postsecondary institution or department could use to 
periodically assess its evolving educational strategies, in particular those 
educational strategies supported by its uses of computing, video and 
telecommunications. 

The FIPSE grant supported the first phase of Flashlight: discussion among the 
five potential "testbed" institutions that would initially be studying their 
strategies, the potential contractor (the Western Cooperative for Educational 
Telecommunications of the Western Interstate Commission for Higher 
Education), and the Annenberg/CPB Project. The discussion's aim was to 
identify those educational strategies and outcomes that are most important to 
monitor. Discussion was carried on initially by e-mail and a Delphi survey; it 
concluded with a two-day working meeting. This report summarizes the 
group's choices of strategies and outcomes. Following the end of the planning 
phase, the Annenberg/CPB Project has decided to support the full 
implementation of the Flashlight Project and to make it the keystone of a new 
Educational Strategies Program. 

Project Director: Stephen C. Ehrmann 

Senior Program Officer 

The Annenberg/ CPB Projects 

901 E Street, NW 

Washington, DC 20004-2037 

202-879-9643 

EHRMANN@SOUL.CPB.ORG 

Project Reports other than this document: 

Proposal to implement Flashlight, April 1994. ... 

"Asking the Right Question: What Does Research Tell Us About Technology and 
Higher Learning?" to appear in the March 1995 issue of Change. 

Requested article now in preparation for the Educom Review. 

Project Presentations: 

Educom '94, AAHE Assessment conference (upcoming), Educom '95 (upcoming) 
and 4 other talks in this country and abroad 
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Project Overview: The Flashlight Project is developing, testing and 
disseminating evaluation procedures that a postsecondary institution or 
department could use to periodically assess its evolving educational strategies, 
in particular those educational strategies supported by its uses of computing, 
video and telecommunications. 

The FIPSE grant supported the first phase of Flashlight: discussion among the 
five potential "testbed" institutions that would initially be studying their 
strategies, the potential contractor (the Western Cooperative for Educational 
Telecommunications of the Western Interstate Commission for Higher 
Education), and the Annenberg/CPB Project. The discussion's aim was to 
identify those educational strategies and outcomes that were most important to 
monitor. Discussion was carried on initially by e-mail and a Delphi survey; it 
concluded with a two-day working meeting. This report summarizes the 
group's choices of strategies and outcomes. Following the end of the planning 
phase, the Annenberg/CPB Project has decided to support the full 
implementation of the Flashlight Project and to make it the keystone of a new 
Educational Strategies Program. 

Purpose: Why "Flashlight?" The goal of this planning activity was to discover 
whether many different types of postsecondary institutions were in fact using 
comparable educational strategies to deal with some of their most serious 
problems, and using technologies in a comparable way to implement those 
strategies. [By "strategy" we refer to patterns of teaching and organizational 
characteristics that are meant to help attain certain goals important to a 
department or institution, e.g., educate graduates who can do certain things, 
enroll and retain certain types of students, etc.] If different types of institutions 
were indeed beginning to use similar educational strategies because of their 
expanding use of similar technologies, then we should be able to develop a small 
yet widely useful set of evaluative procedures. 

The purpose of this planning grant was to see whether our teams would agree 
that their institutions were beginning to solve comparable problem s using 
comparable strategies because of their ripening investments in a small number of 
common technologies. 

Background and Origins: The Annenberg/ CPB Project of the Corporation for 
Public Broadcasting sponsored a program of seven projects in 1990 that 
demonstrated how to develop and institutionalize "New Pathways to a Degree." 
Each used computing, video and telecommunications to support a rich, 
accessible degree path for students learning off-campus. As the program and its 
evaluation were reaching their conclusion in 1993, it had become clear that the 
evaluative issues inherent in New Pathways were complex and general. 
Meanwhile I had been giving a series of invited talks on what is known about 
how best to use information technology to improve postsecondary education. 
When the Maricopa Community Colleges invited me to help lead a district wide 
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staff retreat on the subject of technology, education, and evaluation, these factors 
came together in the idea of Flashlight. We would seek funding to develop and 
test the kinds of evaluative procedures that Maricopa and the New Pathways 
institutions needed. Later Washington State University joined the group. 

Project Description : The FIPSE grant supported the first phase of Flashlight: 
discussion among the five potential "testbed" institutions that would initially be 
studying their strategies, the initial development institution (the Western 
Cooperative for Educational Telecommunications of the Western Interstate 
Commission for Higher Education), and the Annenberg/CPB Project. The 
discussion's aim was to identify those strategies and potential outcomes that are 
most important to monitor. The discussion was initiated with e-mail 
conversation and then a detailed Delphi survey by the author that sketched out, 
element by element, a draft set of evaluation procedures. After two rounds of 
feedback in the Delphi cycle, the planning project concluded with a two-day 
working meeting and the writing of a detailed proposal for implementing 
Flashlight. 

Results: Flashlight should develop five families of procedures, we decided. The 
first three should help institutions monitor their ability to deal with a Triple 
Challenge facing most institutions of postsecondary education: 

1) the need tolmprove learning outcomes for people completing courses 
" "of study; “ 

- 2) the need to extend enrollment and retention; and 

3) the need to control the spiraling costs of education. 

Outcomes are important to monitor, but one also needs to know the reasons for 
success and failure. Thus it is important to study changing strategies — not just 
what the institution's announced intentions are, but how its behavior and 
structure are actually changing. Therefore Flashlight will also develop 
evaluative procedures to monitor changes in 

4) teaching-learning strategies that are especially well supported by uses 
of computing, video and telecommunications and 

5) changes in faculty roles and attitudes. 

Aside from countless changes in the details of the model resulting from the 
Delphi process and the meeting, two major changes were made. The "costs" 
procedures were added and a contractor identified to develop them, and 
proposed procedures to assess the degree of student access to needed 
technologies were dropped. 

Summary: In addition to the modifications of the model summarized above, the 
grant helped convince the Annenberg/CPB Project that Flashlight was worth its 
full support. Flashlight has now become the heart of a new Educational 
Strategies Program whose goal is to help postsecondary education use 
technology effectively to implement the kinds of educational strategies needed to 
improve outcomes, extend access, and control spiraling costs — the Triple 
Challenge. 
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I. Proifct Flashlight - Overview 

Institutions of postsecondary education have begun to invest vast sums in 
computing, video and telecommunications in hopes that this will help them cope 
with the triple challenge of improving educational outcomes, extending 
equitable access, and controlling rising costs per graduate. But there is little 
consensus as to how this is to happen, and little information for each institution 
about whether it is happening and, if not, why not. 



IT. Purpose 

Institutions are investing enormous amounts of scarce dollars in computing, 
video and telecommunications, and so are their students. One major reason for 
this investment is to make possible changes in their educational strategies, e.g., 
more emphasis on students using technological tools to work on realistic, open- 
ended projects; more emphasis on vivid illustrations for lectures; a more far- 
reaching program of distance learning. By "strategy" we refer to patterns of 
teaching _and organizational characteristics that are meant to help attain certain 
goals important to a department or institution, e.g., educate graduates who can 
do certain things, enroll and retain certain types of students, etc. 

Many educators, learners, and benefactors urgently need to know whether their 
investments in technology actually are supporting changes in their strategies, 
and, if so, whether these changed educational strategies are having the desired 
results. Because global statements about "what works" are of limited utility and 
validity, and because the local situation changes on a regular basis, there is no 
substitute for local evaluation of one's own strategies: how are they really 
changing? What are they really accomplishing? . 

Doing an educational evaluation is like using a small, dim flashlight to find one's 
way in a large dark cave. The relative brightness (rigor) of the flashlight 
(evaluation) is much less important than where one points it (asking the right 
question). Any evaluative procedure is designed to answer some specific 
question in a particular way. It is useless to ask “How successful is technology in 
improving education" because that curiosity is too broad to translate into a 
meaningful evaluative study. Thus, in order to help institutions make the most 
(and understand the most) of their investments in technology for education it 
was crucial to first become very specific about what they needed to know. 

The goal of this planning project was to discover whether five very different 
postsecondary institutions were in fact using comparable educational strategies 
to deal with some of their most serious problems, and using technologies in a 
comparable way to implement those strategies. If that were true, then it should 
be possible to develop a limited set of evaluative procedures that would be of 
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wide usefulness. We wanted to see whether we could agree on a set of outcomes 
limited enough and specific enough to provide the foundation for such a set of 
evaluative procedures. 
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tit. background and Origins 

The Annenberg / CPB Project sponsored a program of seven projects in. 1990 that, 
demonstrated how to develop and institutionalize "New Pathways to a Degree. 
Each used computing, video and telecommunications to support a rich, 
accessible degree path for students learning off-campus. 

The Western Cooperative for Educational Telecommunications (WCET) of the 
Western Interstate Commission for Higher Education (WICHE) won the 
competition to evaluate New Pathways. By the time the program and its 
evaluation reached their conclusion in 1993, it had become clear that e 
evaluative issues reached beyond New Pathways and that the WCET study 
only begun to explore them. Meanwhile I had been giving a series of invited 
talks on what is known about how best to use information technology to improve 
postsecondary educations When the Maricopa Community Colleges invited me 
to help lead a district staff retreat on the subject of technology, education and 
evaluation, -these factors came together in. the idea of Flashlight. We would seek 
funding for a project to develop and test the kinds of evaluative procedures that 
Maricopa and the New Pathways institutions needed. Initial discussions were 
held with a number of parties and, as a result, three of the New Pathways 
institutions, Maricopa, and Washington State University joined with WCET and 
the Annenberg / CPB Project to begin planning Flashlight. All five of these 
institutions were already interested in increasing their investment in program 
evaluation and were eager for better tools to do that job. 



These five distinguished and distinctively different institutions of higher 
education were: 

• one of the largest community college districts in the country (Maricopa 

Community Colleges), 

• a public institution that offers a state-wide, virtual community college 

program supported by a combination of video, computing, and 
telecommunications (University of Maine, Augusta - UMA); 

• a major land grant institution with innovative programs exploiting 

technology for students on- and off-campus (Washington State 
University -WSU); 

• an institute of technology with a national record in both distance 

learning and services for the handicapped (Rochester Institute of 
Technology -RIT; and 
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• a public university that exemplifies institutional partnership at 

virtually every level (Indiana University - Purdue University at 
Indianapolis - IUPUI). 



The consultants to the planning activity were Sally Johnstone and Robin Zuniga 
of the Western Cooperative: the team that had lead the New Pathways 
evaluation. Also providing valued advice was Trudy Banta, Vice Chancellor r 
Institutional Planning and Improvement at IUPUI and one of the nation s exper s 
on program evaluation. 



tv. Protect Description 

After a small initial meeting of the WICHE staff, Trudy Banta of IUPUI and this 
author, a Delphi process was begun to help the group prepare for its main 

working meeting. 

In a Detohi process, a group of individuals are each sent a set of questions or 
possibilities^^ this case a set of possible outcomes and practices that each might 
(ormight not) be included in the Flashlight procedures that would be developed 
for their institutions to use; as well as some questions about ‘he basic structom 
and process of the Flashlight Project. These surveys are appended. The feedback 
oh the surveys was complemented by a continuing e-mail conversation on a 
private Internet discussion group (listserv) hosted by Washington State ^ 
University We wanted to identify a set of technology investments changes in 
practice and organization that they enabled, and intended results of the changed 
practice and organization. We also needed to pinpoint a few elements of this 
strateev that were the key points to watch. If we identified too few points or the 
wrong points, the resulting evaluative procedures would not produce the right 
data to guide debate and decision. If we picked too many points, the res g 
evaluative procedures would be too expensive and cumbersome to administer. 

We were designing our flashlight. 

The responses to this second Delphi round were used to create an agenda for an 
intensive two-day workshop in Boulder, Colorado. Our discussion continued 
^til the group had articulated a multi-faceted educational strategy that, they all 
agreed, their institutions were beginning to use, and that was so crucial 
expensive and risky as to require the expense of periodic evaluation. We also 
agreed on a small subset of points within this strategy that were the crucial issues 
that had to be periodically monitored and evaluated. The strategy and the points 
chosenfor evaluation ar/described in the next section of this report. Durmg and 
following the meeting, each institution created a separate action plan, which 
became part of the Flashlight proposal. Each plan described the institution s 
current evaluative work, and how the Flashlight tools would be initially 
employed. These plans have already been submitted to FIPSE as part of the hall 
proposal. (Note: FIPSE decided not to fund Flashlight, so the Annenberg/CPB 
Profect has funded it instead. The Project is now underway, and he five 
institutions have continued to develop and modify their plans. Flashlight 
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procedures are being developed during the spring and summer of 1995; initial 
testing will occur in Fall 1995.) 

V thf. pt anntng protfcTs Conclus ions 

A, Criteria for Developing and I mplementing Strategy 

Before developing our specifications in detail, the Flashlight team first had to 
agree on some ground rules. "Educational strategies" have little or no objective 
existence They are constructs that we use to conceptualize what practitioners 
are doing and should do. We had to decide what sort of construct was worth ou 
e fort We record our criteria here because, if you engage m a strategic planning 
of Evaluation process of your own, you too might find such criteria useful. 

1. It is self-defeating to begin by thinking about what the newest 
technology can do best. As you will see, the applications of techno ogy 
describedhere are neither especially new nor, in technological terms, especia y 
exciting That's just the point. One cannot build cutting edge education on cutting 
^ Sno logy. Cutting edge technology is ordi^rily too expe^e (rehhve to 
its lesser price several years later), too brittle, and too difficult to learn to use. 
That's the bad news. - 7 - --- . .. _ . . - 

The good news is that every year. brings-new "old hat"Technologies. Through 
extensive use in and outside education, these applications of computing, video 
and telecommunications have become affordable, reliable, familiar (almost to the 
point of invisibility), and invaluable. 

2 Focus only on the most important challenges that institutions and 
courses of study must meet in order to fulfill their missions - challenges w.th 

taneible consequences for the institution. Having decided to focus on 

educational, rather than technological, opportunities, our teams agreed that their 
institutions face a Triple Challenge of immediate and obvious urgency. 

i ?im century Wning outcomes: these colleges and universities need to 
hdp their graduates lead^tter lives, especially in ways that also 
support and enrich the communities in which their graduates will 

live. 1 

0 accessibility: these five institutions need to enroll, retain and graduate 
more, and different types of students, increasing the equity of 
educational opportunity; 



i In this essav the term "graduate" will denote anyone who has completed a 
course of study. In a community college, for example, students may complete 
cm irse of studv and receive a certificate. 



3. costs per graduate : these institutions must somehow meet the first two 
challenges and control their costs per graduate in an austere fiscal 
environment. 

This Triple Challenge is an important target because institutional success or 
failure in meeting each challenge can result in gains or losses in financial and 
political support. If the institution begins to succeed, resulting gains could be 
used to institutionalize new practices, to reward faculty, and so on. We'll 
describe this Triple Challenge in more depth later on. 

3. In order to deal with such challenges, one must focus on programmatic 
teaching-learning practices and organizational structures because nothing less 
can change programmatic learning outcomes, accessibility and costs. 
Improvements in single assignments or even in single courses almost never have 
an impact on the average graduate. Such isolated improvements may affect a 
few students a lot, or many students a tiny bit, but rarely do changes in a single 
assignment (thanks to new software) or course meaningfully affect ultimate 
learning outcomes, access chances, or costs -- the Triple Challenge. 

In contrast, institutionalized practices can have a measurable, predictable impact 
on the capabilities of graduates, on accessibility, and on costs. For example, 
some colleges seem to be suffused with values that promote inquiry; their 
bachelor's degree winners go on to earn Ph.D.s in disproportionate numbers, 
decade after decade (Hardy, 1974). .Some institutions have distance learning 
structures and practices that make them exceptionally able to enroll and educate 
working adults. 



4. Ironically, when it comes to technology, attention usually focuses either 
on hardware (computer clusters, the Internet, etc.), on particular pieces of 
software that often are only good for one or two assignments. Thus we decided 
to focus only on uses of technology that could suffuse a student's education by 
making possible pervasive changes in teaching-learning practice and the 
organization of education. 

In other words we decided to focus on a three element vision of strategy: 

institutional patterns of use of technology — > change in organization of learning - 
-> improvements in learning outcomes, access and costs on a departmental or 
institutional scale. 

Technology does not affect the Triple Challenge issues of outcomes, access and 
costs directly. Technology creates new options and constraints for 
administrators, faculty and students; the key is how they individually and 
collectively respond to those new possibilities. That's what Flashlight needs to 
illuminate -- that and the consequences of their responses. 



5. We also had decided beforehand to discuss only those changes that could 
and should be evaluated on a regular basis. The history of educational 
innovation is marked by periodic calls for revolution followed by systematic 
inattention to whether practice is really changing and outcomes are really 
improving. Our institutions are unlikely to be able to make a sustained effort to 
improve education unless they are continually monitoring progress and 
identifying barriers impeding that progress. Even the effort to decide which data 
to gather should provoke constructive conversation among faculty members and 
administrators, discussions about the missing links. 

We commend this set of ground rules to institutions reviewing this report; they 
seem to have a value of their own, even beyond the conclusions we drew by 
using them. 



B. An Emergent, Technology-Enabled Strategy for Responding to the 
Triple Challeng e 

We clarified, with the aid of those ground rules, a strategy that is already being 
pursued by all five of the participating institutions. We believe that many other 
institutions are pursuing their own versions of this same strategy. 

Not all elements of this strategy (technology; changes in organization.of learning; 
changes in the Triple Challenge) will be evaluated by the proposed Flashlight 
instruments. The following narrative will indicate which- particular elements will 
be targeted by the procedures to be developed during the next phase of work 
(1995-1996). 

1. Which Technologies, and Why? 

Our group concluded that there are five important patterns of investment in 
technology and related infrastructure needed for a strategy that would fit our 
ground rules: 

1) Use of hardware, software and technology-based learning resources 
that come from the worlds of work and research. Such ivorldivare 2 
includes personal computers and camcorders, spreadsheets and 
sophisticated molecular modeling software, on-line control of 
interlibrary loan and the Internet. None of these technologies were 
designed for instructional use, and none are marketed primarily for 
instructional use. However, worldware has become the dominant 
type of technology in the curriculum, primarily because of its 
usefulness as a working tool for faculty and students. Student 



2 See Morris et.al.,1994. This EDUCOM-sponsored study of valuable viable 
software coined the term "worldware," details the reasons why this software is 
so widely used and so long-lived, and analyzes the history of several examples 
of worldware. 



editions of worldware (software or hardware developed for 
instructional purposes that strongly resembles worldware) is also 
of great use. 

2) Electronic mail and computer conferencing. Asynchronous 

technology can support discourse that proceeds at a slower and 
more thoughtful pace than in the face to face classroom where 
students have only a second to decide whether and how to respond 
to a faculty member's question. Electronic mail can also be swifter 
and more efficient than the exchange of homework written on 
pieces of paper. FAX machines and voice mail are also useful for 
these purposes, especially for students studying "off-campus." 

3) For those institutions serving students who study off-campus, it's 

important to provide video and audio transmission of live and 
prerecorded presentations , as well as real-time conferencing for 
small groups of scattered individuals (these days audio- and 
audiographic conferencing seem most cost-effective). These same 
systems can also be used to include new instructors and off-campus 
experts, and they can help on-campus students get more from 
lectures, but the investment is usually justified by its obvious 
benefits of increased enrollment. - - - "• _ 

4) Networked educational partnerships enable institutions (including 

government, research laboratories, libraries, schools, and 
businesses) to share resources and services electronically. 

Examples include systems of interlibrary loan and coordinated 
purchasing based on the assumption of such a system of sharing 
information. The University of Maine at Augusta, one of our five 
institutions, is part of a university-school network that ties colleges 
and schools together by a video, audio and computer network. The 
colleges can offer classes that are electronically available at high 
schools; the high schools provide a broader array of services for 
their rural communities and use their facilities more hours of the 
day, and days of the week. 

5) Policies and practices to assure that all students in a course of study 

have access to the hardware, software and materials they need for 
their coursework. If only 90% of the students can get the 
technology they need, faculty can't confidently require students to 
use technologies to do assignments; thus our five institutions, and 
others like them, have devised a mix of policies to increase student 
access to technology. 

Networked educational partnerships can sometimes aid in 
providing technology access; the Educational Network of Maine 
enables college students to use computers, fax machines, and its 
network by coming to nearby high schools. 



To repeat, these five types of investment in technology are not being made 
merely for their own sake. Our five institutions, and others like them, are 
making such investments in order to enable the following kinds of instructional 
practices and organizational changes. The group ultimately decided not to 
develop measures of these investments, not because such measures would not be 
valuable but simply because our resources for developing and testing evaluative 
procedures would be limited. 



2. Changes in the Organization of Learning 

The technological investments identified above are worth making (and 
monitoring) because they make the following teaching practices and 
organizational changes more feasible (and thus should help the institution 
respond to the Triple Challenge). The changes in teaching practice include more 
emphasis on complex projects, collaboration, and the pace of academic 
conversation. The changes in organizational structure include structures to 
support study off-campus and changes in faculty roles. 



a) Changes in Teaching and Learning Practices 

What elements’ of teaching and learning are most important>to promote? We 
were interested in improvements that would help meet the Triple Challenge and 
we had our ground rules (e.g., the practice had to be capable of being 
institutionalized so that it would pervade a student's education). After long 
discussion, faculty-administrator teams from our five institutions agreed on 
several points. 

First we decided that all such teaching practices would have to foster greater 
engagement in learning and more productive time on task by students. There 
are two ways to do this: 

1) by attracting students to spend more and better quality time in study 

and 

2) wasting less of their time (e.g., in commuting). 

As Astin (1993) indicates, virtually all research indicates that, the more time 
students spend studying, the more they learn. 

Our teams agreed that several teaching-learning strategies meet all our goals and 
ground rules, and are thus worth specific evaluative monitoring. These short 
summaries describe them, and indicate how the five institutions' investments in 
technology make these practices more feasible: 

• Project-based learning, i.e., student learning by working on complex, 
open-ended, realistic (or real world) projects. Doing scientific 
research, composing music, writing an interpretive essay, and 
diagnosing a case are all examples of project-based learning. 
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Worldware can provide students with crucial tools and resources 
for tackling such projects, e.g., statistical software for data analysis, 
computer aided composition and performance for music, on-line 
research libraries for scholarship, and, of course, word processing 
for large written assignments. 

Off-campus students should and can learn by working on such 
projects, too. For example, students from all across Maine can 
register for a meteorology course, studying at nearby high schools 
which are linked: a) by video to all of Maine's public college and 
university campuses , b) by computer to a unified catalogue of 
libraries across the state, and c) by computer to the global Internet. 
These students, many in the rural north of Maine, can analyze 
same-day weather data and images downloaded from the United 
States Weather Service via the Internet. 

• Collaborative learning. Our teams all pointed to the increasing 

importance of learning in teams at their institutions. They also 
value learning to communicate and work across cultural barriers, 
something becoming increasingly common these days. 

The ease with which students work in teams around computers is 
well-documented: their projects and even their, problems seem to • 
pull them together. Worldware “enables courses to focus on the kind 
of large real-world problems that in turn require te~amwork to be . 
resolved. Electronic mail and computer conferencing enable students 
who live some distance from one another and from campus to 
collaborate. 

• Learning at paces and times of student's choosing. Traditional 

education can be constraining, especially in seminars (where 
students have to respond quickly or not at all) and on commuter 
institutions (where group work and lab work have to be done 
during narrow windows of time when students are on campus). 

Our institutions' five investments in technology are, in contrast, 
liberating. For example videotaped lectures can be rewound and 
reviewed as often as needed. Electronic mail offers an alternative 
pace for intellectual exchange. Audioconferencing makes it possible 
for small groups or teacher-student conferences to occur at a wide 
range of times, even when one of the parties is traveling. The 
worldware- based projects can be done wherever there are 
computers. 

• Learning marked by continuous improvement of a piece of work. 

Computer-based projects such as essays are mechanically easier to 
revise, giving both faculty and students an opening to rethink the 
work. The FIPSE Technology Study Group report, Ivory Towers, 
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Silicon Basements (1988) spotlighted this growing phenomenon 
almost seven years ago, and called it "doing it again, thoughtfully." 
Our institutions want to find out how commonly faculty are 
building revision and rethinking of work into course syllabi and, as 
we'll see, whether there are observable consequences in the skills of 
graduates. 

• Improved student-faculty and student-student interaction, and 

enhanced feedback. Faculty from our institutions have noticed 
that project-based, collaborative learning seems to stimulate 
interaction and feedback. Electronic mail lowers barriers of role 
(teacher-student) and ethnicity that can sometimes interfere with 
honest communication even in conventional classrooms. Electronic 
mail is proving especially important where the interaction is 
between people from different cultural backgrounds. For example, 
it has been often observed how readily Native American students 
open up when conversations are carried on by electronic mail. 

(e.g., Arias and Bellman, 1990) 

Each of these elements will be targeted by Flashlight procedures for monitored 
changes in teaching-learning practices in degree programs and across 
institutions. 

b) Structures for Education in Virtual Space - - 

Using technology to extend and enrich education simultaneously is nothing new. 
Consider the printed book: a technology that 1) opens learning to new students 
while 2) giving each student access to more, richer resources. The book, to be 
successful, requires increasing the distance between the student and the master, 
to give the student time and space to reflect. The book also increases the author- 
master's reach over distance, and across time. 

Recently there has been explosive growth in this use of newer technologies to 
further enrich education while increasing distance among students, faculty 
members and the resources of study. This is sometimes called educating 
students in "virtual space" but in colleges and universities today it is far more 
routine than that futuristic label implies. When students learn in virtual space, 
each of the four facets of the teaching-learning process can include more 
students, while giving each student more control and more resources: 

1) Direct instruction, offered via live or pretaped video. Use of video is 

good for students, and not just because it means that more students 
can learn. The lecturer can be rewound at will if the student has a 
tape of the presentation. The faculty member can include primary 
source video from outside the institution. The public colleges and 
universities in Utah, Oregon and other states use statewide video 
networks, for example, while the Rochester Institute of Technology 
(RIT) prepares videotaped lectures in advance and lease students 
the cassettes. 



Institutions need not always produce their own video 
presentations. In the world of video and computer courseware, the 
Annenberg/CPB Project is an interesting case in point. Most of its 
course materials were developed at costs of $2-$7 million each. 
Interestingly enough the cost per student to develop these materials 
is relatively low because their use is so widespread (an estimated 2 
million students a year use Annenberg/ CPB materials). When 
these course materials are adopted as the foundation of a course, 
substituting for locally developed lectures, faculty members are 
freed to focus more attention on student projects, discussion, and 
individualized support. 

2) Technology-based tools and resources for "learning by doing (and 

reflecting)" are also becoming available in virtual space, most 
notably the on-line library catalogue of an international "collection" of 
resources. Some institutions, such as the University of Maine 
System, then mail students the library materials they request from a 
multi-institution collection. Shared library catalogues are a good 
metaphor for the larger problem of supporting "learning by doing” 
on a large scale. Such support requires organizational change, in 
this case, new infrastructure that enables libraries to work together. 
The Internet is often seen as an environment where unbridled 
spontaneity is sufficient to support learning. "Need pen pals from 
French-speaking countries for your students? Use the Internet!" 

That works fine for a few students, but- to support large numbers of 
learners, universities will need to develop stable, networked 
educational partnerships with other institutions, partnerships that 
benefit all participating parties. The development of such large 
scale partnerships (college-college, college-school, college-business, 
and so on) is one of the next challenges facing educators. 

3) Conversation in real time for small groups is more frequently 

available by audio and audiographic conferencing. Organizationally, 
institutions need to support student use of equipment; the 
Rochester Institute of Technology (RIT) makes extensive use of 
these technologies, and has a system for leasing students needed 
equipment for the duration of the course. 

4) Time-delayed exchange of conversation and academic work can be 

carried on through electronic mail and fax. The networked 
educational partnership in Maine illustrates how students in 
distant rural areas can be given access to needed computers, 
modems and fax machines. 

Supporting these four types of interaction for students in a large number of 
courses requires not only technology but also organizational changes. Once 
made, these changes also enable support of richer, more accessible student 



services, from the research library, to counseling and financial aid, to that hard- 
to-define but crucial service known as supporting "academic community" for a 
far-flung and diverse student body and instructional staff. 

Although we assume that institutions using Flashlight procedures will be using 
virtual space, and we intend to help them monitor the outcomes of this effort, we 
decided against immediate development of measures of change in practice in this 
area, e.g., measures of the extent to which all four conversations are supported or 
other indicators of how virtual space is being used for teaching and learning. 
Once again, such measures could be of significant value, but we knew we would 
lack the resources for their development at this time. This area is a priority for 
future funding. 



c) Implications for Faculty and their Work 

To make these changes happen, our five teams pointed to important changes 
needed in patterns of faculty work. Here are some of the faculty issues they 
hope to monitor on a regular basis: 

• "Positive addiction" by faculty to teaching in the transformed setting. 3 

Faculty long ago became positively addicted to paperback books 
and photocopying; they rely on them for good reason, and would 
find it painful to be deprived of them, especially if their students 
still had to take the same exams and do the same term projects. 
Our teams want to monitor positive addiction by faculty and their 
students to the five key investments in technology. 

• To what extent are their faculty changing their roles from "sage on the 

stage" to "guide on the side?" 

• To what extent are faculty members getting constructive feedback on 

their teaching, thanks to use of technologies such as videotaped 
lectures and electronic mail? 

In addition to those priority issues, our institutions are also interested in 
monitoring: 

• Any shift in faculty work style from "lone ranger" in classroom to "team 

member" working with instructional and student support 
personnel; 

• Incidence of faculty who feel burned out by technology, and faculty 

whose careers are revitalized. 



3 For a fuller description of "positive addiction" as an indicator of how 
thoroughly faculty or students have come to depend on information 
technology, see Ehrmann(1991). 



At this writing, we intend that our procedures will help institutions monitor their 
status in the first three areas; it may not be possible to do so for the latter two. 



3. Resulting Responses to the Triple Challenge 

To sum up, our five institutions, different as they are, are using their own 
variations on those five investments in technology (worldware, computer 
conferencing, presentations and real-time conferencing for students off-campus, 
networked educational partnerships, and universal access to needed technology) 
to foster three families of strategic changes in: a) teaching and learning practice, 
b) structures to support learning in virtual space, and c) faculty work. Each of 
those strategic changes in practice and structure is in turn meant to help them 
meet the Triple Challenge of providing 21st century learning, expanding access, 
and dealing with austerity. 



Investments in 
Technology 


Changes in Teaching- 
Learning and 
Organization 


The Triple Challenge 


Worldware 


Project-based, 
collaborative learning, 
more flexibility, and . 
more feedback 


Need for 21st century 
skills 


Computer conferencing 
& e-mail 


Presentations and real- 
time conferencing for 
students off-campus 


Structures for education 
in virtual space 


Increasing accessibility to 
education for a diverse 
population of adults 


Networked educational 
partnerships 


Universal student access 
to needed technologies, 
networks 


Changes in faculty work, 
e.g., shift from sage on 
stage to guide on side, 
shift from Lone Ranger 
to team leader. 


Controlling costs per 
graduate (relates to all 
investments in tech, all 
change in practice and 
structure) 



Let's return to the Triple Challenge (column three in the figure, above) and see 
how these changes in practice and structure (made more feasible by strategic 
investments in technology) can help institutions make tangible progress. It turns 
out that each of the new practices has implications for all three of the Challenges. 
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a) The Challenge of Educating Adults for the 21st Century 

Change in the organization of work is increasing demand for higher level skills 
in all workers. The 1991 report by the Labor Secretary's Commission on 
Achieving Necessary Skills (SCANS) concludes, "...good jobs will increasingly 
depend on people who can put knowledge to work" (U.S. Department of Labor, 
1991) All workers need to acquire proficiency in basic skills (reading, writing, 
computation, listening, speaking) but almost everyone also needs to be able to 
think creatively, collaborate, and adapt readily to changes in their work, 
including technological changes. (Reich, 1991; Commission on the Skills of the 
American Workforce, 1990, Carnevale et.al, 1988) Arguably the most important, 
most neglected skills are those needed to work in groups and coalitions 
(Boyatzis, 1982; Klemp, 1977) Nor or all these teams in the work world going to 
be composed of people of all the same gender, race and cultural background. 
Learners need to create and function within teams with diverse members. 

Our five teams agreed that it was most important to foster the following four 
"21st century abilities" and then monitor their use by graduates who are now at 
work or in further education: 

• Ability to use information technologies to learn, think, rethink (revise) 

and act in their fields; 

• Ability to apply college learning to problems of their current work 

(content mastery, basic skills), including ability to identify 
appropriate tools and information (including the use of Internet); 

• Ability to identify a learning need and how to deal with it, especially 

where learning involves use of information technology; and 

• Ability to form, work in, and lead teams 

b) The Challenge of Raising the Doorway: Equitable Access 

Colleges want to educate all the students they can (within the constraints of their 
goals, mission, and selectivity), but the pool of potential learners has grown. Its 
composition has shifted toward adults who are distant from the campus offering 
the program they need, who have schedules that clash with those of the faculty 
and with other students', who are physically challenged, and who have 
extremely varied preparation and learning styles. In fact a "new majority" of 
today's current students fall into this category (Pew..., 1990). Still more such 
adults would like to enroll but are prevented from doing so. 

The problem is not simply one of disadvantage (a label that points the finger of 
responsibility and blame at the learner and the learner's home community) but of 
accessibility. Imagine a school with doorways 5' 5" feet high. It would soon have 
two populations of students: normal students and head- injured (nontraditional) 
students, otherwise (privately) called "weeds." Some faculty would believe in 



providing each weed with a raft of compensatory medical and educational 
services, while others would argue that this special treatment would be unfair to 
normal students (those who coincidentally were less than 5' 5" high). However, 
there is a third option: invest in raising the height of the doorways. Once the 
brain-damaged weeds graduate, there would be only one type of student: 
normal. 

We think of many types of students today as disadvantaged or marginal because 
of what is wrong with our educational institutions, not with them. Traditional 
technologies of learning have only been available at one place at one time, and 
could serve limited numbers of students; anyone else, no matter how motivated 
or capable, who couldn't learn inside the cloistered walls, was defined as 
nontraditional. The five investments in technology, and the changes in practice 
and structure they make possible, should enable institutions to attract, retain and 
graduate many types of students previously on the margins of higher education. 
(Western Cooperative..., 1994) 

Our five teams think it is particularly important (and feasible) to help more 
students enroll, persist, and graduate — adults who otherwise wouldn't because 
of the following six issues: 

• location 

• family/work responsibilities 

• physical disability 

• being a non-native speaker of English (there is evidence that computer 

conferencing enables such students to excel because they have 
more time to interpret what they hear and to compose what they 
want to say. (Hiltz, 1989; Arias and Bellman, 1990) 

• varied learning styles (e.g., students who might not enter, persist or 

excel without aids such as computer conferencing, the ability to 
"rewind" a lecture, and the ability to take computer-administered 
practice tests) 

• economic disadvantage. 

In each of these six areas, our teams would like to monitor progress year by year, 
testing whether the spotlighted changes in practice and organizational structure 
are aiding equity of access. 




c) Controlling Costs per Graduate: The Challenge of Austerity 

Colleges and universities are competing with health care, programs for the aging, 
and other growing elements of the public sector for a share of the nation's 
discretionary spending. 

There have been many promises that technology would cut educational costs, 
presumably by substituting itself for faculty members. That hasn't happened 
and probably won't. However, our five institutions do see the possibility of 
controlling costs per graduate. The most likely mechanisms include: 
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• possible savings in capital costs and operating expenses, and possible 

increases in income, gained by offering a high quality education 
offered wholly or partially in virtual space rather than in expensive 
brick-and-mortar facilities. The entire Educational Network of 
Maine cost less to create than one new high school; 

• increased enrollment and retention (resulting in decreased costs per 

graduate). Retention should be aided by project-based 
collaborative learning (engaging instruction) and by education in 
virtual space (less wasted time, resulting in high course loads and 
quicker graduation); and 

• stronger public and alumni/ ae support, due to improved performance 

(measured and publicized by assessing the performance of 
graduates) and to participation in networked educational 
partnerships, so that cogent appeals for resources can be based on 
the needs of the larger partnership. 

We decided to tackle the first job of learning to monitor the first two issues, but 
not the third one at this time. 

VI. Summary and Conclusion: "The Parable of the Ships at Sea" 

There is nothing new under the sun. _Elting Morison 4 recounts what happened to 
the United States Navy when, in the middle of the last century, it collided with 
advances in its basic technologies, during a period when the nation and its 
military faced new challenges. 

Up to the middle of the nineteenth century, nations relied on warships that were 
constructed of wood, powered by sail, and armed with guns that could shoot 
neither far nor accurately. As Morison put it admiringly, the ship of the line 
"was a demonstration of the use of limits... The designers intended to build a 
machine that would do as much work as possible within the restricting scales, 
proportions, tempos, strengths of materials, and structural simplicities imposed 
upon them by the knowledge and means available to them." 5 Furthermore, 
because the technology of the ship had changed little over the centuries, neither 
had the strategy and tactics for using those ships. 

Those technologies changed dramatically and permanently midway through the 
nineteenth century: the old ways were challenged by steam power, iron hulls, 
and rifled cannon of greater size and power. But what should such ships do, and 
what should they look like? 




4 Morison, Elting E., From Know-how to Nowhere: The Development of American 
Technology, New York: Basic Books, 1974. 

5 Ibid, p. 150. 
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